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Background / Context: 

Description of prior research and its intellectual context. 

Access to Head Start has been shown to improve children’s preschool experiences and school 
readiness on selected factors through the end of 1st grade. Two more years of follow-up, through 
the end of 3 ld grade, can now be examined to detennine whether these effects continue into the 
middle elementary grades. The statistical design and impact analysis methodology for the Head 
Start Impact Study (HSIS) has been lauded as exemplary for a large scale social experiment, and 
analytic strategies continue to evolve through three rounds of analysis covering six years of 
sample follow-up (U.S. Department of Health and Human Services, 2005, 2010). 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

Based on a randomized admission lottery in hundreds of local sites, the HSIS provides a platfonn 
for measuring Head Start’s impacts — both short tenn and long tenn — on a nationally 
representative sample of participants without selection bias, providing internally and externally 
valid impact estimates for the overall Head Start participant population and for subsets of 
participants of particular policy interest. The proposed presentation will describe the study’s 
path-breaking design and innovative analytic methods and share what the study team has learned 
about the sustained effects of Head Start as children complete 3 rd grade. It will also explain the 
methods and sample used to determine these impacts. 

Setting: 

Description of the research location. 

(May not be applicable for Methods submissions) 

The HSIS includes a nationally representative sample of 4,667 three- and four- year-old children 
in 383 centers nationwide. The evaluation has followed sample children through the end of 3rd 
grade, with rich data available across many outcomes over multiple years. All newly-entering 
children in the fall of 2002 entered the admission lottery and were divided in a 6:4 ratio into 
treatment group members admitted to Head Start and control group members who were 
excluded. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features , or characteristics. 

(May not be applicable for Methods submissions) 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

(May not be applicable for Methods submissions) 

Head Start provides developmentally-focused pre-school education to children from 
disadvantaged families with the aim of ensuring their Kindergarten readiness in cognitive, 
behavioral and social domains. 

Significance / Novelty of study: 

Description of what is missing in previous work and the contribution the study makes. 
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Unlike any prior research project, the HSIS randomized children applying for Head Start entry in 
2001 into treatment and control groups to ensure unbiased impact estimation. Treatment group 
members were granted access to Head Start in the first year while members of the control group 
were not. 

Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 

A method of computing impacts, developed in earlier reports for lower grade levels, is refined 
and reapplied here and emphasis placed on academic and developmental outcomes of particular 
importance to children in 3 rd grade. Several major statistical issues are given in-depth treatment 
at the cutting edge of randomized control trial (RCT) impact analysis methods: crossovers from 
the control group into Head Start, lagged collection of background data in interviews and child 
assessments conducted several weeks after program entry, controlling the family-wise error rate 
when tests for program impacts are conducted on dozens of outcomes in four different years, 
interpretation of thousands of subgroup impact estimates (also in a multiple-comparison- 
controlled analytic framework), repeated-measures trend analysis, and many state-of-the-art 
measurement and psychometric components. 

Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or real data. 

Each of the analytic issues just noted has challenged RCT impact analysts in studies beyond the 
HSIS over the last decade, and many methodological innovations of the HSIS have been 
presented to the field in conferences and publications (e.g., Bell et al. 2003). Some have already 
proven their robustness to other applications beyond the HSIS, and much more of this “methods 
knowledge” transfer seems likely in the future. The proposed presentation will point out these 
spillover benefits of the HSIS research, concentrating particularly on methods of experimental 
impact analysis with strong salience to educational interventions of interest to SREE. 

Research Design: 

Description of the research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, 
analytic essay, randomized field trial). 

(May not be applicable for Methods submissions) 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

(May not be applicable for Methods submissions) 

Findings / Results: 

Description of the main findings with specific details. 

(May not be applicable for Methods submissions) 

Conclusions: 

Description of conclusions, recommendations , and limitations based on findings . 

The foundation of random assignment both empowers and challenges methodological 
development when seeking to measure program effectiveness without bias and with balanced 
interpretation as to the strength of the statistical data and measurement tools used. Throughout 
its 12-year course, the HSIS has sought to capitalize on its experimental design, larger 
representative sample, and vast rich data elements. This caldron of ingredients has yielded a 
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number of analytic advances of interest to other researchers focused on measuring the 
effectiveness of educational interventions. Findings on effectiveness through 3 rd grade will also 
hold strong interest to the SREE research community. The extraordinarily high follow-up rates 
(i.e., low data collection attrition) achieved by the study over six years make these impact 
estimates particularly compelling and the policy ramifications of the study highly germane to 
national debates about the direction of the federal Head Start program 
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Appendices 

Not included in page count. 


Appendix A. References 

References are to be in APA version 6 format. 
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